Facial Phoneme Extraction for Taiwanese Sign Language Recognition

We have developed a system that recognizes the facial expressions in Taiwanese Sign Language (TSL) using a phoneme-based strategy. A facial expression is decomposed into three facial phonemes, including eyebrow, eye, and mouth. A fast method is proposed for locating the areas facial phonemes. The shapes of the phonemes were then matched by the deformable template method, giving feature points representing the corresponding phonemes. The trajectories of the feature points were tracked along the video image sequence and combined to recognize the type of facial expression. The tracking techniques and the feature points used have been tailored for the facial features in TSL. According to each special need, different tracking techniques were applied to different facial phonemes. We regard the eyebrows as a rigid object and assume the inner corner and the outside corner of an eye are two fixed points in an image sequence. Therefore, the template matching